Dental shade mapping

ABSTRACT

A method and apparatus for generating a color mapping for a dental object. The method includes generating a transformation matrix according to a set of spectral reflectance data for a statistically valid sampling of teeth. Illumination is directed toward the dental object over at least a first, a second, and a third wavelength band, one wavelength band at a time. For each of a plurality of pixels in an imaging array, an image data value is obtained, corresponding to each of the at least first, second, and third wavelength bands. The transformation matrix is applied to form the color mapping by generating a set of visual color values for each of the plurality of pixels according to the obtained image data values and according to image data values obtained from a reference object at the at least first, second, and third wavelength bands. The color mapping can be stored in an electronic memory.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a Divisional of U.S. Ser. No. 12/834,921 filed onJul. 13, 2010 entitled “DENTAL SHADE MAPPING”, in the names of Wong etal, incorporated herein in its entirety.

FIELD OF THE INVENTION

This invention relates generally to methods and systems for dental colormeasurement and more particularly relates to a digital method and systemfor determining color shade information for natural teeth, referenceshade samples, and fabricated dental prostheses.

BACKGROUND OF THE INVENTION

Modern restorative dental procedures often require accurate colormatching, such as for filling materials and for the fabrication ofrestorations such as crowns, implants, fixed partial dentures, andveneers. The materials used for these procedures, such as ceramics andother materials, can be skillfully formed and treated to closely matchthe shape, texture, color and translucency of natural teeth.

A widely used technique for determining and communicating tooth colorinformation is a process referred to as “shade matching” whereby thedentist or technician visually matches a patient's tooth to one of anumber of reference shade samples or shade tabs within one or more setsof standardized shade guides. The practitioner who performs the matchrecords the identification of the matching shade tab and conveys thatinformation to the dental laboratory where the restoration or prosthesisis then fabricated. The laboratory then uses its own set of the sameshade guides to perform visual color evaluations of the restoration orprosthesis throughout the fabrication process.

The visual shade matching process can be highly subjective and subjectto a number of problems. The initial matching procedure is oftendifficult and tedious, and it is not unusual for the process to taketwenty minutes or longer. In many cases, there is no shade tab thatperfectly matches the patient's teeth.

The problem of accurately modeling the color of a tooth is more complexthan obtaining a close color match using shade tabs. The inherentshortcomings and limitations of both instrument-based and visual-basedshade-matching systems can be more fully appreciated by considering thedifficulties involved in matching the appearance of human teeth. Toothcolor itself results from a relatively complex interaction ofreflection, transmission, refraction, fluorescence, and scattering by avariety of organic and inorganic components. It is influenced byvariations in tooth pulp volume, dentin condition, enamel composition,and other variations in the composition, structure, and thickness of thedental tissues. One result of this complexity is that color appearanceand color measurement are greatly influenced by lighting geometry,spectrum, surrounding colors, and other environmental factors.

As a further complication, color within a single tooth is generally notuniform. Color non-uniformities can result from spatial variations incomposition, structure, thickness, internal and external stains, surfacetexture, fissures, cracks, and degree of wetness. As a result,measurements taken over relatively large areas produce averaged valuesthat may not be representative of a tooth's dominant color. In addition,natural color variations and non-uniformities make it unlikely that agiven tooth can be matched exactly by any single shade tab. This meansthat a method for conveying the distribution of color within a tooth,not just its average color, is required. Further, tooth color is seldomuniform from tooth to tooth. Therefore, the ideal color of a restorationmay not be in visual harmony with that of an adjacent tooth or of anyother single tooth in a patient's mouth. Moreover, people generally areparticular about the appearance of their teeth. Understandably, they arequite intolerant of restorations that appear inappropriate in color.

In cosmetic dentistry, the fabrication lab often requires additionalinformation in order to more accurately map tooth color in addition tosimple shade matching. In practice, the dentist or technician mayprovide a photograph in addition to a shade tab, so that the fabricationlab can adjust color characteristics over different portions of thetooth. This helps to provide a type of color mapping for subjective use,with information that relates to the shade tab and shows how colors inother portions of the tooth vary from that of the shade tab.

It is often difficult to decide which tab matches most closely (or,conversely, which has the least mismatch) and to provide accurateinformation on color variation over the tooth surface. Frequently, thepractitioner determines that the patient's teeth are particularlydifficult to match, requiring that the patient go in person to thelaboratory that will be fabricating the restoration. There, trainedlaboratory personnel can perform the color match and color mapping. Inmany cases, the patient may even need to return to the dentist andlaboratory two, three, or even more times as the color of the prosthesisis fine tuned by sequential additions of ceramics or other coloredmaterials. In a high percentage of cases, estimated to be nearly 10% forsome dental prostheses, the visual color matching procedure still failsand the prosthesis that has been fabricated is rejected for color orvisual harmony by the dentist or by the patient.

Considering the relative difficulty of the color matching task, and thefurther complexity of color mapping, a high rate of failure is not atall surprising. Visual color evaluation of relatively small colordifferences is always difficult, and the conditions under which dentalcolor evaluations must be made are likely to give rise to a number ofcomplicating psychophysical effects such as local chromatic adaptation,local brightness adaptation, and lateral-brightness adaptation.Moreover, shade tabs provide at best a metameric (that is, non-spectral)match to real teeth; thus, the matching is illuminant-sensitive andsubject to variability due to normal variations in human color vision,such as observer metamerism, for example.

In response to the need for improved color matching and color mapping indental applications, a number of approaches have been attempted. Somesolutions to this problem are of the following general types:

(i) RGB-based devices. With this approach, an image of the entire toothis captured under white light illumination using a color sensor.Tristimulus values are calculated over areas of the tooth surface fromRGB (Red, Green, Blue) values of the 3-color channels of sensor, makinguse of a color calibration transform. Color analysis by RGB-baseddevices relies heavily on the quality of the captured image and requiresrobust calibration and may require use of the same camera forcolor-matching of tooth and prosthetic device. This requirement can bedue to calibration of the camera itself as well as to colorpreprocessing that is performed within the camera in order to providethe RGB data; this preprocessing can vary significantly from one camerato the next, even for cameras from the same manufacturer. Maintainingaccuracy tends to be difficult due to metamerism, in which the colormeasured is highly dependent upon the illuminant. This is particularlytroublesome since dental measurement and imaging are generally carriedout under conditions that differ significantly from natural lightingconditions. Examples using RGB measurement and employing a correspondingcolor transform in this way include: U.S. Pat. No. 5,766,006 entitled“Tooth Shade Analyzer System and Methods” to Murljacic; U.S. Pat. No.6,008,905 entitled “Method and Apparatus for Determining the Appearanceof an Object” to Breton, et al.; and U.S. Pat. No. 7,064,830 entitled“Dental Color Imaging System” to Giorgianni et al.

(ii) Colorimetric devices. Devices of this type are engineered todirectly measure color as perceived by the human eye. With this type ofdevice, illuminating light or reflected light (under white lightillumination) is filtered at the three wavelength bands that correspondto the spectral response characteristic or color matching functions ofthe eye, and measured reflected signals are directly translated intotristimulus values. As with RGB-based devices described in (i),measurements from this type of device also suffer from metamerism. Someexamples using this approach include those disclosed in U.S. Pat. No.5,383,020 entitled “Method and Apparatus for Determining the Color of aTranslucent Object Such as a Tooth” to Vieillefosse that requires aspectrometer and U.S. Pat. No. 6,867,864 entitled “Optical MeasurementDevice and Related Process” to Overbeck et al.

(iii) Spectrophotometric devices. These devices employ spectralreflectance for obtaining color data. Illuminating or reflected light isspectrally scanned, and light reflected by the tooth is recorded, usinga photosensor, as a function of wavelength. Visual color, that is, CIE(Commission Internationale de L'Éclairage or International Commission onIllumination) tristimulus color information, is then calculated from themeasured spectral reflectance curve. Spectrophotometric devices are notsubject to the same tendency to metamerism inherent to colorimetric andRGB-based devices and, potentially, yield more accurate colormeasurements. It is significant to note, however, that thespectrophotometer is not an imaging device. The spectrophotometer is aninstrument that measures the spectral content of incoming light over asmall area using a photosensor. Examples of tooth color measurementusing spectrophotometric devices include U.S. Pat. No. 4,836,674entitled “Method and Apparatus for Determining Color, in Particular of aDental Prosthesis” and U.S. Pat. No. 6,038,024 entitled “Method andApparatus for Determining the Color Stimulus Specification of an Object”to Berner.

Although the data obtained using the spectrophotometric approachprovides advantages for more accurate color matching over colorimetricand RGB approaches, including elimination of metamerism, this approachhas been found difficult to implement in practice. The use of a lightscanning component for measuring different spectral components,generally employing a grating or filter wheel, tends to make thespectrophotometric system fairly bulky and complex. This makes itdifficult to measure teeth toward the back of the mouth, for example.Attempts to alleviate this problem have not shown great success. As oneexample, U.S. Pat. No. 5,745,229 entitled “Apparatus for DeterminingOptical Characteristics of an Object” to Jung et al. provides a compactspectrophotometric device employing optical fibers to channel reflectedlight to an array of sensors, each sensor using a different spectralfilter. However, as is true of spectrophotometric devices in general(iii, above), this device measures only a small area of the toothsurface at a time. To obtain a color mapping of an entire tooth surfacerequires numerous separate measurements with this approach. The imagecapture process is time-consuming and does not provide consistentresults. Color mappings can be inaccurate using such an approach, sincethere can be considerable sensitivity to illumination and image captureangles and probe orientation during the imaging process.

In general, conventional methods that employ color filters, either atthe illuminant end or at the sensor end, can be less desirable becausethey are subject to the limitations of the filter itself.

Thus, there is a need for an improved measurement apparatus thatprovides dental shade matching and mapping in a procedure that isstraightforward to execute, having a high degree of accuracy, butwithout high cost or complex components.

SUMMARY OF THE INVENTION

An object of the present invention is to advance the art of color shademapping in dental applications. With this object in mind, the presentinvention provides an apparatus and method that obtains spectralreflectance data from multi-color images of the tooth without thecomplexity of using a spectrophotometer.

An advantage of the present invention is that it employs an imagingarray for obtaining spectrophotometric measurements over the full image.This makes embodiments of the present invention readily adaptable forintraoral camera use. In addition, the output spectrophotometric colordata that is provided is not subject to metamerism, which affectssolutions that use colorimetric and RGB color matching techniques. Theapproach used in embodiments of the present invention obtains spectralreflectance data for each pixel of the tooth image, allowing accuratemapping of tooth color, including variation in color over differentportions of the tooth.

These objects are given only by way of illustrative example, and suchobjects may be exemplary of one or more embodiments of the invention.Other desirable objectives and advantages inherently achieved by thedisclosed invention may occur or become apparent to those skilled in theart. The invention is defined by the appended claims.

According to one aspect of the invention, there is provided a method forgenerating a color mapping for a dental object, executed at least inpart by a control logic processor, comprising: generating atransformation matrix according to a set of spectral reflectance datafor a statistically valid sampling of teeth; directing illuminationtoward the dental object over at least a first, a second, and a thirdwavelength band, one wavelength band at a time; obtaining, for each of aplurality of pixels in an imaging array, an image data valuecorresponding to each of the at least first, second, and thirdwavelength bands; applying the transformation matrix to form the colormapping by generating a set of visual color values for each of theplurality of pixels according to the obtained image data values andaccording to image data values obtained from a reference object at theat least first, second, and third wavelength bands; and storing thecolor mapping in a computer-accessible electronic memory.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other objects, features, and advantages of theinvention will be apparent from the following more particulardescription of the embodiments of the invention, as illustrated in theaccompanying drawings. The elements of the drawings are not necessarilyto scale relative to each other.

FIG. 1 is a schematic diagram showing the arrangement of components in aconventional image-based apparatus for obtaining a color measurement fora tooth.

FIG. 2 is a schematic diagram showing an earlier method that makes useof a color calibration transform for obtaining tooth color values.

FIG. 3 is a schematic block diagram that shows a color mapping apparatusaccording to one embodiment of the present invention.

FIG. 4A is a graph showing intensity measurements related to specificwavelengths.

FIG. 4B is a graph showing typical spectral characteristics for lightsources used in one embodiment.

FIG. 4C is a graph showing a spectral response characteristic for abroadband sensor array in one embodiment.

FIG. 5A is a schematic diagram showing the tristimulus data obtained foreach pixel using conventional color matching.

FIG. 5B is a schematic diagram showing the spectral reflectance dataobtained for each pixel using the apparatus and methods of the presentinvention.

FIG. 6A is a logic flow diagram of steps for obtaining visual colormapping data from a tooth.

FIG. 6B is a logic flow diagram of steps for obtaining spectralreflectance data from a tooth in an alternate embodiment.

FIG. 7 is a logic flow diagram showing a process for forming atransformation matrix according to one embodiment of the presentinvention.

FIG. 8 is a logic flow diagram showing a process for forming atransformation matrix according to an alternate embodiment of thepresent invention.

FIG. 9 is a logic flow diagram showing a process for forming atransformation matrix according to another alternate embodiment of thepresent invention.

FIG. 10 is a logic flow diagram showing a process for forming atransformation matrix according to yet another alternate embodiment ofthe present invention.

DETAILED DESCRIPTION OF THE INVENTION

The following is a detailed description of the preferred embodiments ofthe invention, reference being made to the drawings in which the samereference numerals identify the same elements of structure in each ofthe several figures.

In the context of the present application, the term “narrow band” isused to describe LED or other illumination sources that emit most oftheir output light over a narrow range of wavelengths, such as 20-50 nmwide. The term “broadband” is used to describe a light sensor thatexhibits high sensitivity to incident light over a wide wavelength rangeextending at least from about 400 nm to about 700 nm. Because this typeof sensor responds to light but does not distinguish color, it is oftenreferred to as a “monochrome” sensor or, somewhat inaccurately, as a“black-and-white” sensor.

In the context of the present application, the term “pixel”, for “pixelelement”, has its common meaning as the term is understood to thoseskilled in the image processing arts. An electronic image of an objectis captured by an array of light-sensitive elements, each of whichprovides the signal for forming a pixel of image data.

Figures shown and described herein are provided in order to illustratekey principles of operation and component relationships along theirrespective optical paths according to the present invention and are notdrawn with intent to show actual size or scale. Some exaggeration may benecessary in order to emphasize basic structural relationships orprinciples of operation. Some conventional components that would beneeded for implementation of the described embodiments, such as varioustypes of optical mounts, for example, are not shown in the drawings inorder to simplify description of the invention itself. In the drawingsand text that follow, like components are designated with like referencenumerals, and similar descriptions concerning components and arrangementor interaction of components already described are omitted. Where theyare used, the terms “first”, “second”, and so on, do not necessarilydenote any ordinal or priority relation, but are simply used to moreclearly distinguish one element from another.

The terms “color” and “wavelength band” are generally synonymous as usedin the context of the present disclosure. For example, a laser or othersolid-state light source is referred to by its general color, such asRed, rather than by its peak output wavelength (such as 635 nm) or itswavelength band (such as 630-640 nm).

The term “set”, as used herein, refers to a non-empty set, as theconcept of a collection of elements or members of a set is widelyunderstood in elementary mathematics. The term “subset”, unlessotherwise explicitly stated, is used herein to refer to a non-emptyproper subset, that is, to a subset of the larger set, having one ormore members. For a set S, a subset may comprise the complete set S. A“proper subset” of set S, however, is strictly contained in set S andexcludes at least one member of set S.

In the context of the present disclosure, the term “dental object”refers to an object, material, or other element for intra-oral use orapplication and includes teeth, prosthetic devices such as crowns,dentures, braces and other supports and bridges, filling materials,shade-matching tabs, and the like. In contrast to some attempts atcharacterizing tooth color, the apparatus and methods of the presentinvention take into account a combination of factors that affect colormeasurement and that complicate the task of accurately characterizingcolor. Particular embodiments of the present invention identify andcompensate for variable factors such as illumination wavelengths anddetector response characteristics in order to derive accurate colordata. To do this, the approach that is used in embodiments of thepresent invention obtains, for each pixel in the image of a tooth orother dental object, spectral reflectance data that is substantiallyindependent of the spectral response of the measurement device and thatcan be used to provide an objective measure of color that applies forilluminant over any combination of wavelengths. As a result, the datathat is obtained for tooth color mapping in the present invention can beused to reconstruct the visual color of an object when viewed under anyilluminant taken from a set of available illuminants with known spectraldistributions. The color mapping that is generated for a dental objectcan then be used for generating a displayed image or used for comparisonagainst color mapping data for another object or material, such as acrown or other dental prosthetic device or a filling material. The colormapping that is generated for a dental object can also be used fordesigning and forming a dental prosthetic device, for example. A colormapping for a tooth or other dental object can consist of a considerableamount of data and is typically stored as a data file in acomputer-accessible memory.

Referring to FIG. 1, there is shown a schematic block diagram of aconventional imaging apparatus 10 for obtaining dental color data. Anillumination source 12 directs light onto a tooth 20. A capture module18 then performs image capture and provides image data to a colorreconstruction module 22. The output is a set of visual color valuesthat correspond to points on the tooth surface.

The basic arrangement of FIG. 1 is used for each of the image-basedcolor measurement approaches described earlier in the backgroundsection. FIG. 2, for example, shows a schematic diagram for implementingthe system described in the Giorgianni et al. '830 patent cited earlier.In an imaging apparatus 30, illumination source 12 provides white lightillumination to tooth 20. A lens 32 directs reflected light to a sensor34 that provides corresponding Red, Green, and Blue values for eachpixel. A color calibration transform 36 then generates visual colorvalues as output for every image point.

In contrast to the approaches shown in FIGS. 1 and 2, the apparatus andmethods of the present invention provide a color mapping apparatus thatutilizes color reconstruction based on spectral reflectance. Thisapproach is advantaged over RGB-based and colorimetric devices byproviding an inherently more accurate set of data on the actual colorcharacteristics of the tooth. Advantageously, this method is not subjectto metamerism, which would otherwise render measurements dependent onthe illuminant of the measuring system. Unlike spectrophotometricmeasurement devices in general, the apparatus and method of the presentinvention obtain spectral data for each pixel in the tooth image.Moreover, this information is obtained using an imaging array ratherthan a photosensor. The imaging array used in embodiments of the presentinvention is a monochrome sensor, although a color sensor canalternatively be used.

The apparatus and methods of the present invention obtain not only anaccurate color measurement for a single pixel or grouping of adjacentpixels, but, because they use an imaging device, also provide suitableinformation for accurate color mapping over the full image of the dentalobject. By obtaining spectral measurement data using an imaging array,embodiments of the present invention obtain measurements that allowspectral data to be generated for each pixel of the tooth image.

Referring to the schematic diagram of FIG. 3, a color measuring andmapping apparatus 40 uses an illumination apparatus 24 that can providelight of separate colors. In one embodiment, illumination apparatus 24consists of multiple narrow band light sources 14 b, 14 g, 14 y, and 14r, shown as color LEDs, having wavelength bands λ1, λ2, λ3, and λ4,respectively. In the embodiment of FIG. 3, four LEDs are shown by way ofexample; there can be any number of different colors and more than asingle LED or other light source for each color. A broadband imagingsensor array 44, a CCD (charge-coupled device) or CMOS (ComplementaryMetal-Oxide Semiconductor) imaging array in this embodiment, provides aset of output values for each pixel corresponding to reflection fromeach narrow band light source. A reference target 28 is an object thatis optionally provided for obtaining reference intensity measurementsused in correcting for intensity fluctuations in the system, asdescribed subsequently. In one embodiment, reference target 28 is a graypatch, with known spectral reflectance characteristics. Reference target28 and tooth 20 are both within the field of view of imaging lens 32. Inan alternate embodiment, reference target 28 is an object that does notappear in the image field, but is a separate device used for obtainingreference and calibration data.

In order to obtain the spectral data using the apparatus shown in FIG.3, the LEDs or other light sources can be energized according to colorgroups, one color group at a time in rapid succession, and thecorresponding measurements of reflected light are obtained by imagingsensor array 44. As shown in thumbnail form in FIG. 3, this effectivelyprovides, for each pixel of the tooth image, a number of points on agraph, one intensity reading corresponding to each of wavelength bandsλ1, λ2, λ3, and λ4 in this example. A statistics-based transformationmatrix 50 is used to generate the full spectral reflectance curve usingthese points. The result is a spectral reflectance curve for eachindividual pixel of the tooth image. These data provide a color mappingthat more accurately represents a color shade as compared to othermethods that simply attempt to measure tristimulus values directly orperform color conversion from RGB to a standard color space, such ashue-saturation-brightness value (HSV) or Commission Internationale deL'Éclairage L*a*b* (CIELAB) color space, for example. Also shown in FIG.3 is control logic processor 38, which performs control logic processingand contains supporting computer-accessible electronic memory componentsthat execute these processing functions and store interim and finalresults. Such components are familiar to those skilled in the imagingarts.

The graph of FIG. 4A shows an example in which four discrete intensitymeasurements are obtained for a pixel, one for each light source 14 b,14 g, 14 y, and 14 r, corresponding respectively to wavelength bands λ1,λ2, λ3, λ4. This yields four points in a graph of intensity vs.wavelength, as shown.

Taking accurate measurements requires knowledge of different energylevels and measurement sensitivities in the system. As FIG. 4B shows,each narrow-band light source, an LED in this embodiment, provides anoutput intensity over a constrained range. Wavelength bands λ1, λ2, λ3,and λ4 are identified by the nominal wavelengths at which theseintensity curves peak.

Broadband sensor array 44 can be any type of sensor array that measuresthe amount of reflected light that corresponds to the illuminationapparatus 24 component that is energized and that provides a measuredvalue for each pixel. In one embodiment, broadband sensor array 44 has abroad spectral response characteristic over the visible wavelengthrange. This is distinctly different from the “color matching functions”of the eye or the spectral response of a color sensor array, both ofwhich have significant values only in isolated bands of the visiblewavelengths. For example, the monochrome CCD imaging array may have atypical quantum efficiency curve as shown in the example of FIG. 4C.This device measures the amount of reflected light that corresponds toeach light source 14 b, 14 g, 14 y, and 14 r, as it is illuminated andprovides an output signal that is indicative of the sensed lightintensity received. Because the peak wavelength and bandwidth of the LEDlight source is known, the measured value can be readily associated witha wavelength, without the need for a color filter array (CFA) or otherfiltering component.

In contrast to some color-matching solutions, the method and apparatusof the present invention provide a spectrophotometric or spectralreflectance color mapping that effectively stores, for each pixel in thetooth image, estimated spectral reflectance data values R_((tooth)) thatcan be used to accurately profile the color of the tooth surface. Bycomparison against conventional methods that rely on RGB calibrationtransform or colorimetric measurements, the methods of the presentinvention are capable of generating a mapping that includes aconsiderable amount of data for each pixel of the tooth image. This isrepresented schematically in FIGS. 5A and 5B. FIG. 5A shows the colordata gathered for a single pixel P using conventional color-measuringapproaches. In the example shown, a single data value is provided foreach of the Red, Green, and Blue color planes. By contrast, FIG. 5Bshows the nature of the spectral reflectance mapping data that isobtained for each pixel using the methods of the present invention.Here, each pixel P effectively has an associated spectral reflectancecurve that provides a substantial amount of information on its actualcolor content, from which tristimulus values X, Y, Z or other color datacan be derived. In practice, only a small amount of data may actually bestored in memory for each pixel P following this processing; thespectral reflectance curve itself can be reconstructed using the storedmatrix of coefficients obtained as described earlier. In memory storage,then, each pixel can be associated with a set of spectral reflectancevalues and, optionally, also have a link or other identifier for amatrix that is capable of re-creating the full spectral reflectancecurve for that pixel. Alternately, calculated tristimulus values, CIELABvalues, or values in other standard color space could be stored for eachpixel P.

One advantageous result of having spectral reflectance data values isimproved color matching between dental objects, such as between a toothand a prosthetic device or material. Using the data acquisition andprocessing sequence of the present invention, color matching can beaccomplished using any of a number of mathematical techniques. In oneembodiment, the spectral reflectance curve for a dental object A iscompared with the spectral reflectance curve for a dental object B anddifferences between the two curves are evaluated for closeness of fit orother metric. For example, the overlap area between two curves overdifferent wavelength ranges can be evaluated. In other embodiments,tristimulus data values or CIELAB color values are obtained for each ofthe dental objects and are compared.

General theory and principles of obtaining spectral reflectance data forpixels in an image, using multiple narrow-band measurements from amultispectral camera, were demonstrated and described by Peter D. Burnsand Roy S. Berns in an article entitled “Analysis Multispectral ImageCapture”, Fourth Color Imaging Conference: Color Science Systems andApplications, 1996, pp. 19-22. These researchers utilized a white lightsource with seven interference spectral filters, successively switchedinto position, to obtain image data from a set of color referencesamples at different wavelengths. Principal Component Analysis (PCA) wasthen applied to provide a set of scalar values usable for reconstructingspectral reflectance data for an unknown color patch, using a similarlyobtained sequence of color images. Although the work of theseresearchers showed the feasibility of using a small number ofmeasurements for obtaining accurate spectral reflectance data, thesystem employed would be impractical for color mapping of teeth andother dental objects and materials. The design of a multispectral camerautilizing multiple switched interference filters is not compatible withthe size and access constraints of dental imaging.

The logic flow diagram of FIG. 6A shows steps executed by control logicprocessing components in color measuring and mapping apparatus 40 ofFIG. 3 for generating visual color values for a tooth in one embodimentof the present invention. A transformation matrix generation step S100initially executes using spectral reflectance data from thestatistically valid sampling of teeth, as described subsequently. Alooping procedure executes once per source color, here for each LEDcolor or other narrow-band illumination source. In an illumination stepS110, the LED or other narrow-band illumination source is energized. Animage capture step S120 obtains an image of tooth 20 and, optionally, ofreference target 28 at the given illumination. The resulting imageconsists of light reflected from the tooth, captured for each pixel inimage capture step S120. The array of image values measured from thebroadband sensor array is then obtained and stored in an obtain valuesstep S130. The loop consisting of steps S110, S 120, and S130 repeatsuntil each color group of LEDs or other narrow-band light source hasbeen energized. The net result is a set of image values for each pixelof the image for the tooth or other dental object.

Obtain values step S130 results in the set of N signals for N imagewavelength values s={s₁, s₂, . . . , s_(k)}. Alternatively, step S130can include additional signal values based on transformations of theabove signal values, for example polynomial transformations, s={s₁, s₂,. . . , s_(k), s₁ ^(e), s_(k) ^(e), . . . s_(k) ^(e)}, where e is a realnumber. Alternately, the transformed values of the image samplemeasurements can be calculated using a simple power law with theexponent in the range [0.3-0.4].

Continuing with the logic flow in FIG. 6A, a matrix application stepS160 applies a transformation matrix to the measured image values. Anobtain visual color values step S174 generates visual color values fromthe image data according to the transformation matrix. The color mappingthat is generated is then stored in an electronic memory by controllogic processor 38 in a storage step S190. The memory itself can be arandom-access device, for short-term storage, or an optical or magneticstorage unit, for longer-term storage. The color mapping can begenerated as needed, so that memory storage as shown in FIG. 6A is atemporary “workspace”, and is used by control logic processor 38 onlyfor the duration of performing a color match or comparison, or duringdisplay of an image for example. Alternately, storage step S190 canstore a color mapping for a longer term, such as part of a database ofcolor mappings, indexed by illuminant types, for example. Once the colormapping is formed and is available in computer-accessible memory, it canbe used for comparison against color mapping data for a prostheticdevice or material, for example.

The logic flow diagram of FIG. 6B shows expanded steps for generating acolor mapping in one embodiment. Steps S100-S160 are similar to thosedescribed with reference to FIG. 6A. Following application of thetransformation matrix in step 160, a step S170 generates spectralreflectance data or curve using the mixture coefficients obtained instep S 160 and previously computed principal components. A color valueextraction step S180 then extracts visual color values for the colormapping based on the spectral distribution of a desired illuminant.

Advantageously, spectral reflectance for tooth and other dentalmaterials is well-behaved and follows a characteristic pattern. Spectralreflectance curves for teeth and dental materials exhibit a smooth,consistent shape, with variation only within a relatively limited range.This characteristic enables the use of statistical techniques forproviding the tools needed for more accurate color matching than isavailable using conventional approaches.

Generating Transformation Matrix 50

As FIG. 3 showed, transformation matrix 50 is used for providing visualcolor data based on a relatively small number of measured values for atooth. Embodiments of the present invention generate transformationmatrix 50 from a statistically valid sampling of spectral reflectancedata for a large number of teeth. The number of teeth in a“statistically valid sampling” has enough members to be representativeof the larger tooth population. Increasing the sample size beyond astatistically sufficient or statistically valid number of samples tendsto have no noticeable effect on the resulting data that is obtained fromthe sampling distribution.

Referring to FIG. 7, a process for forming transformation matrix 50 ingenerate transformation matrix step S100 is shown. Initially, a sampleset of spectral reflectance curves 52 is obtained using aspectrophotometer. The sample set contains spectral reflectance curves52 for a sufficient number of teeth M for use as a statistically validsample; in one embodiment, M=100, for example. Principal ComponentAnalysis (PCA) shown at 54 is then used to analyze this data to identifythose principal components 56 with greatest statistical importance, andthe corresponding mixture coefficients 60. Empirical data has shown thatthe number of significant principal components for spectral reflectanceof teeth and other dental objects is about 4.

Principal Component Analysis (PCA) is a well-known vector spacetransform technique that is used in statistics to reduce the overallnumber of dimensions for multidimensional data sets in order to bettershow the relationships in the multidimensional data sets. Formultivariate data, such as spectral reflectance data, PCA analyzes thecovariance of data variables in order to help uncover trends in the datathat can be used in order to better understand the data and to simplifytheir use. PCA involves a decomposition of data by generatingeigenvectors (mutually orthogonal in an l₂ norm) that provide analternate basis or coordinate system for the data. PCA determines anordered set of eigenvectors, with the ordering based upon decreasing theeigenvalue associated with the respective eigenvector. Since theeigenvectors can be scaled to unit vectors and are orthogonal, anorthogonal linear transformation from its original coordinate system toan alternate coordinate system of eigenvectors is obtained. With thistype of data decomposition, the low-order components (that is, thefirst, second, third, and subsequent principal components in order)represent the most important aspects of the data; higher ordercomponents represent increasingly less information about the variance ofthe data. This enables statistical characterization of the data in asimpler form (that is, at fewer dimensions) than as the data had beenoriginally acquired.

Referring back to FIG. 7, a set of low-order principal components isgenerated at 56, along with the corresponding set of mixturecoefficients 60. Mixture coefficients are used to scale or weight theprincipal component eignevectors for generating each of the set of teethspectral reflectance curves for sample set of reflectance curves 52.Simply stated, the mixture coefficients form a vector in the vectorspace determined by the PCA eigenvectors.

Still referring to the logic flow of FIG. 7, in one embodiment,transformation matrix 50 is generated by taking image value measurements62 of the M teeth in the sample set using color measuring and mappingapparatus 40 (FIG. 3), at the same locations where the spectralreflectance curves were recorded previously. Four measurements for eachof the M teeth are represented at measurements 62 in FIG. 7; the numberof measurements that are obtained for each tooth corresponds to thenumber of light sources (LEDs) in illumination apparatus 24. Aleast-squares fit procedure 78 is then performed between the set ofmixture coefficients 60 and the measured image data, measurements 62, togenerate transformation matrix 50, which provides the best-fit matrixfor converting measured image data 62 to mixture coefficients 60.

The transformation matrix 50 generation step, according to theembodiment described in FIG. 7, generates a transformation matrix forfuture use in color mapping. This matrix and the set of principalcomponents 56 are then stored in control logic processor 38 of colormeasuring and mapping apparatus 40. Then, each time that color measuringand mapping apparatus 40 is used on an unknown tooth, the storedtransformation matrix 50 is applied to the obtained image values atpixels of the tooth image to calculate a set of mixture coefficients(step S 160), which can then be used with the stored principalcomponents 56 to generate spectral reflectance data (step S170 in FIG.6B). The generated spectral reflectance curve can then be used with anyviewing illuminant's spectral profile and the visual color matchingfunctions to calculate the tristimulus values XYZ of the tooth (stepS180 in FIG. 6B).

In an alternate embodiment for generating a transformation matrix instep S100, as shown in the logic flow diagram of FIG. 8, PCA 64 isperformed on the measured image data 62 for the statistically validsampled set, instead of on the spectral reflectance data 52 from thesample set. Values in measured image data 62 are obtained using mappingapparatus 40 (FIG. 3). The result is the set of significant principalcomponents 66 and the corresponding mixture coefficients 70 for themeasured image value data 62. XYZ color values 63 are obtained directlyfrom each curve of the sample set of spectral reflectance data, measuredusing a spectrophotometer as described earlier. A least squares fitprocedure 78 is performed between the obtained XYZ values 63 and themixture coefficients 70 to generate values for a transformation matrix72. In this embodiment, transformation matrix 72 is the best-fit matrixfor converting mixture coefficients 70, corresponding to the principalcomponents of the measured image data 62, to tristimulus values. Thisembodiment directly calculates the visual color for a predeterminedviewing illuminant; it does not provide complete spectral reflectanceinformation.

FIG. 9 describes another alternate embodiment of step S 100 forgenerating a transformation matrix without performing PCA. XYZ colorvalues 63 are first obtained from each curve of the sample set ofspectral reflectance data, recorded using a spectrophotometer asdescribed earlier. Values in measured image data 62 are obtained usingmapping apparatus 40 (FIG. 3). A least squares fit is then performedbetween the obtained XYZ values 63 and the measured image values 62 togenerate values for a transformation matrix 74. Transformation matrix 74in this embodiment is the best-fit matrix for converting the measuredimage values 62 directly to tristimulus values. It is simpler inimplementation than the previous two embodiments described with respectto FIGS. 7 and 8. But unlike the embodiment in FIG. 7, and similar tothe embodiment in FIG. 8, this sequence yields only XYZ color values fora predetermined viewing condition.

FIG. 10 describes yet another alternate embodiment of step S 100 forgenerating a transformation matrix without performing PCA. XYZ values 63are first obtained from each curve of the sample set of spectralreflectance data, recorded using a spectrophotometer, as describedearlier. Visual color values 200 are computed for each set of XYZvalues. These visual color values can be chosen so that they representcoordinates in an approximately uniform visual color space, such asCIELAB. Values in measured image data 62 are obtained using mappingapparatus 40 (FIG. 3). In this embodiment of step S 130, however, theset of measured image data is expanded to include polynomial-transformedvalues of each measured image value, s={s₁, s₂, . . . , s_(k), s₁ ^(e),s_(k) ^(e), . . . s_(k) ^(e)}, wherein e is a real number, as describedearlier. A least squares fit is then performed between the obtained XYZvalues 63 and the expanded set of measured image values 210 to generatevalues for a transformation matrix 76. Transformation matrix 76 in thisembodiment is the best-fit matrix for converting the expanded measuredimage values 210 directly to visual color values. Unlike the embodimentsin FIGS. 7, 8 and 9, this sequence yields only visual color values for apredetermined viewing condition.

As has been noted earlier, the task of obtaining accurate colormeasurement using conventional methods is confounded by variation ofillumination and sensor response. These problems are addressed by theimproved method of the present invention. The generalized measuredsignal S_(i(tooth)) that is obtained for a particular wavelength λ, forLED illumination in the equations that follow, using color measuring andmapping apparatus 40, can be represented as follows:

S _(i(tooth)) =∫I _(LEDi)(λ)R ₀(λ)D(λ)dλ  (1)

wherein R₀ is the actual reflectance of the tooth or other dentalobject. This variable is also expressed, with reference to the toothmaterial, as R_(tooth); D(λ) is the sensor response; I_(LEA) is theintensity of the i^(th) LED or other narrow-band light source.

It can be observed that equation (1) holds true for color measurement ofthe tooth in general, whether using RGB or white light sources andwhether the image sensor array is a CCD, CMOS, or other type of lightsensing device. It is particularly instructive to note that conventionaltooth color measurement devices, such as RGB-based color measurementdevices and colorimetric measurement devices, generate the tooth colorvalues from measured signal S_(i(tooth)). As equation (1) shows,however, this signal is itself the product of three variable factors:the actual tooth reflectance R₀, the illuminant, and sensor response. Itis due to the dependency of this measurement on the light source and onsensor response that accurate color computation is not obtained with aconventional RGB-based or colorimetric measurement apparatus. However,unlike conventional solutions, the approach of the present inventionisolates tooth reflectance R₀ from the other two factors in the measuredsignal. The obtained reflectance R₀ can then be used to accuratelycharacterize color under any lighting conditions. In this way, themethod of the present invention employs a spectrophotometric approach tocolor characterization. However, unlike conventional spectrophotometricinstruments that obtain one or more measurements to extract this datafor a single photosensor at one location on the object, the apparatusand methods of the present invention obtain this data usingwell-characterized narrow-band light sources with a monochrome sensorarray that captures a mapping of spectrophotometric data for the fullobject.

In embodiments of the present invention, multiple signal valuesS_(i(tooth)) are obtained, one for each of the N LED color light sourcesindexed i (i=1, 2, . . . , N). As equation (1) shows, in order toprovide accurate and consistent measurement using S_(i(tooth)), it isnecessary to compensate for short-term changes in the device, such asfrom fluctuations in illuminant intensity and/or sensor response D(λ).For this reason, as part of the imaging process, a reference targetmeasurement S_(i(ref)) is optionally obtained for use as calibrationreference:

S _(i(ref)) =∫I _(LEDi)(λ)R _(ref)(λ)D(λ)dλ  (2)

wherein R_(ref) is the reflectance of reference target 28. This value isobtained using image readings from reference target 28 (FIG. 3), whichis positioned within the same captured field of view as the tooth in oneembodiment. In an alternate embodiment, image readings from referencetarget 28 are obtained separately, such as at the beginning of animaging session, for example.

Given values S_(i(tooth)) and S_(i(ref)), the more useful quantity forimplementing the methods of the present invention is the correctedmeasurement value as given by:

$\begin{matrix}{{\hat{S}}_{i} = {\frac{S_{i{({tooth})}}}{S_{i{({ref})}}}R_{{ref}{(\lambda)}}}} & (3)\end{matrix}$

wherein R_(ref(λi)) is the known reflectance of the reference target 28at the peak value of LEDi. The corrected measurement value Ŝ_(i) removesfirst-order sources of measurement variability and eliminates dependenceon the scale of intensity or device response. From the correctedmeasurement value Ŝ_(i), the method of the present invention can be usedto obtain visual color values to provide a highly accurate colormapping.

Each of the tristimulus color values X, Y, Z (hereafter represented byX_(q) (q=1, 2, 3), where X₁=X, X₂=Y, and X₃=Z) for each pixel can becalculated from the tooth reflectance in the following way:

X _(q) =∫I(λ)R _(tooth)(λ) x _(q)(λ)dλ  (4)

wherein the value of x _(q)(λ) is the corresponding visual colormatching function of the standard observer, and I(λ) is the spectraldistribution of the light source under which the tooth is viewed.

Value R_(tooth)(λ) is not directly measured by color measuring andmapping apparatus 40, but can be obtained using mixture coefficientsthat are derived from analysis of the sample set, such as using PCA, asdescribed previously.

Spectral reflectance measurement results obtained from color measuringand mapping apparatus 40 can be used to calculate color tristimulusvalues for tooth under any lighting condition with known spectral energydistribution, according to equation (4). Applying this procedure to bothteeth and shade-matching tab will yield two sets of tristimulus valuesfrom which the best match can be found. Alternatively, the tristimulusvalues can be converted to a different visual color space, such as theCIELAB or HSV color space, for finding the closest match between toothand shade-matching tab.

The above approach has been described for a monochrome broadband sensorarray. The same method can also be used if a conventional RGB colorsensor array is used to provide a signal as each LED or otherillumination source is energized. In the case of an RGB color sensor,the color channel with the highest signal level for a particular LEDillumination can be used. If all three color channels of the sensor areused for each LED illumination, measured image data values will beobtained from a combination, such as a weighted sum, of the signals ofthe three color channels.

Using embodiments of the present invention, the task of obtaining aspectrophotometric color mapping of the tooth surface is simplified, andthe cost of providing this data is significantly reduced overconventional alternatives. A result of applying the transformationmatrix of the present invention is that only a few measurements areneeded in order to obtain highly accurate color data. Measurements fromillumination using only three or four LEDs has been found to besufficient for color shade mapping in many intraoral imagingapplications.

It is noted that the sequence of FIGS. 6A and 6B allow color mappingunder any of a set of viewing illuminant conditions. The spectralreflectance mapping data that is collected can be used to calculate thecolor that is best matched with any source of viewing light, includingincandescent light, fluorescent light, natural light or sunlight, orother source. Thus, for example, the spectral reflectance mapping datathat is obtained in a dentist's office can be used to determine the bestmatched color for prosthesis viewed in natural light (sunlight), or forprosthesis viewed under stage lighting, such as for a model orperformer, or for prosthesis viewed in fluorescent lighting in officeenvironment, or other lighting conditions. This is an advantage of themethod of the present invention over earlier RGB and colorimetricmethods in which the color mapping and color match were constrained tothe specific lighting conditions used in making the color measurement,which could be very different from the lighting condition that mattersmost to the patient.

The information obtained from the process of FIGS. 6A and 6B can bestored and used in a number of ways. The visual color values that aregenerated can be directly used to render the mapping for display, forexample, or can be encoded in some way and stored in memory. Spectralreflectance values can be correlated with other tooth image data andprovided to a dental laboratory or other facility for which accuratecolor characterization is useful.

It is noted that the method described in FIGS. 6A and 6B obtainsmultiple measurements for each pixel of the tooth image in oneembodiment. This is potentially a sizable amount of data, but provides afull characterization of the spectral content of the tooth image, pixelby pixel. The data collected for the tooth image is non-metameric, thuseliminating the undesirable effects of illumination dependence that cancompromise the color data when conventional colorimetric, RGB, orvisually matched systems are employed for color mapping.

LEDs are advantaged as light sources for obtaining spectral reflectancedata according to the present invention. In a preferred embodiment, theLED emits most of its light over a range of wavelengths that is lessthan about 40 nm. In one embodiment, the LED sources are substantiallynon-overlapping with respect to their respective wavelength bands, sothat any leakage of light into adjacent bands is negligible. Other typesof narrow band light sources could alternately be used to provide theneeded illumination for reflectance measurements. Alternate types oflight source include filtered light from a broadband light source, suchas a lamp, for example. Four LEDs are shown in the example of FIG. 3. Ingeneral, at least three light sources of different wavelengths should beused.

Because the apparatus and methods of the present invention provide amapping of spectral reflectance values, they promote accurateinformation about the true color of the tooth than is available whenusing conventional colorimetric or RGB-based measurement methods.Because spectral reflectance contains complete color information,independent of illuminant, the color data that are obtained are notsubject to error due to metamerism. By using LEDs or other smallsources, without the need for gratings or other devices, the apparatusof the present invention can be packaged in a compact fashion, at lowcost. For example, color measuring and mapping apparatus 40 can bepackaged as an intraoral camera.

Reference target 28 can be any suitable type of reference object, suchas a patch, for providing a reference image for color imaging. A whiteor gray patch can be used, as well as some other patch having uniformspectral content. Target 28 can be separately imaged, or it can bepositioned alongside the tooth, at approximately same distance from andwithin the imaging field of view of sensor array 44, such as in a fixedposition, or may be pivoted into position, such as by energizing anactuator, for reference imaging as needed in the imaging cycle.

A number of computation functions are employed, such as for obtainingtristimulus values and for storing and displaying results, for example.It can be appreciated that these functions can be provided by a controllogic processor 38 provided with or configured to interact with thecolor matching and mapping apparatus of the present invention. Storedinstructions, for example, configure the processor logic circuitry toexecute the color mapping data access, calculations, and outputfunctions described above. Any of a number of types of control logicprocessor devices, such as a dedicated computer workstation, personalcomputer, or an embedded computing system that employs a dedicated dataprocessing component (such as a digital signal processing component)could be used for computation functions. Control logic processingapparatus access an electronic memory for data storage and retrieval. Anoptional display device can also be provided for displaying color matchand color mapping results.

The method and apparatus of the present invention employ a sensor array34 as the detector in order to obtain a mapping with highly accuratecolor information. This component can include an arrangement of multipleCMOS or CCD sensors, typically assigned one per pixel. In oneembodiment, a broadband monochrome sensor array is used. However, it isalso possible to employ a sensor array that is configured for R, G, Bcolor sensing, or configured for some other color space characteristics.Methods of obtaining spectral reflectance data in the present inventioncan be similarly applied to such devices, as has been discussed earlier.It is noted that pixel spacing can be varied for color matching, so thatmultiple sensor sites in sensor array 44 are grouped or clusteredtogether, such as to obtain an averaged value, for example. Because thecolor measurement apparatus of the present invention uses an imagingsensor array, the same device that is used for conventional intra-oralcolor imaging can be configurable for both imaging and color measurementmodes of operation. Referring to the schematic block diagram of FIG. 3,for example, color imaging can be performed by changing the pattern ofillumination from the illumination apparatus 24, the resolution of thesensor array 34, and the color processing provided from control logicprocessor 38. A mode switch (not shown) or mode control command issuedfrom an operator interface can be provided in order to set the operatingmode of apparatus 40 for either conventional imaging or tooth colormapping as described herein.

Illumination apparatus 24 of the present invention employs multi-colorLEDs in one embodiment. However, other light sources that can providemultiple color illumination could alternately be employed, includingother types of solid-state light sources or more conventional lamps orlamps equipped with color filters.

Initial and periodic calibration of color measuring and mappingapparatus 40 are needed in order to compensate for component aging anddrift, so that the profile of each LED in illumination source 12 can bemaintained and regularly updated.

The invention has been described in detail with particular reference toa presently preferred embodiment, but it will be understood thatvariations and modifications can be effected within the spirit and scopeof the invention. The presently disclosed embodiments are thereforeconsidered in all respects to be illustrative and not restrictive. Thescope of the invention is indicated by the appended claims, and allchanges that come within the meaning and range of equivalents thereofare intended to be embraced therein

1. An apparatus for obtaining a color mapping for a dental object,comprising: an illumination apparatus energizable to direct illuminationof at least a first, a second, and a third wavelength band, onewavelength band at a time, toward the dental object; a monochrome imagesensor array having broad spectral response over at least the first,second, and third wavelength bands, that is disposed with respect to anoptical system and that is actuable to capture an image of the dentalobject at each illumination wavelength band to form a set of images ofthe dental object; a reference target that is disposed in the objectfield of the optical system and substantially in focus with respect tothe image sensor array; and a control logic processor operativelyconnected with the illumination apparatus and monochrome image sensorarray and responsive to stored instructions to energize the illuminationapparatus to sequentially provide the at least first, second, and thirdwavelength bands and further responsive to stored instructions tocapture and store an image at each wavelength band, to calculate imagedata values for each pixel, and to generate and store a color mappingaccording to a sample set of spectral reflectance data and according tomeasured image values from the sample set.
 2. The apparatus of claim 1wherein the apparatus is in the form of an intraoral camera.
 3. Theapparatus of claim 1 wherein the control logic processor is adapted to:generate a set of tristimulus values from the set of spectralreflectance data; obtain a set of image sample measurements by directingillumination toward each tooth over the at least first, second, andthird wavelength bands, one wavelength band at a time, wherein the setof image sample measurements are obtained at the same locations on theteeth as the set of spectral reflectance data; generate a plurality ofmixture coefficients by applying principal component analysis to the setof image sample measurements; form a transformation matrix using a leastsquares fit between the plurality of mixture coefficients and the set oftristimulus values; and apply the transformation matrix to form thecolor mapping.
 4. The apparatus of claim 1 wherein the control logicprocessor is adapted to: generate a set of visual color values from theset of spectral reflectance data; obtain a set of image samplemeasurements by directing illumination toward each tooth over the atleast first, second, and third wavelength bands, one wavelength band ata time, wherein the set of image sample measurements are obtained at thesame locations on the teeth as the set of spectral reflectance data;form a transformation matrix using a least squares fit between the imagesample measurements and the set of visual color values; and apply thetransformation matrix to form the color mapping.